Combining Timbric and Rhythmic Features for Semantic Music Tagging

The recent explosion of online streaming services is a reflection of the high importance the multimedia has in our everyday life. These systems deal with large collections of media pieces and their usability is tightly related to the meta-data associated with content. As long as the meta-data is correctly assigned users can easily reach what they were looking for. The presence of poorly annotated data and continuous addition of new items can lead to poor recommendation performances and loss of earning because a large part of the collection will remain unexplored . Automatic tagging can be a step toward the solution for this problem, however state of the art system are not yet ready to substitute human annotators, expecially for music. In this thesis we propose a novel approach to semantic music tagging. The project takes inspiration from Hidden Markov Models and exploits a statistical framework to semantically link two acoustic features. We make the assumption that acoustically similar songs have similar tags. We model our collection of known songs as a graph where the states represent the songs and the transition probabilities are related to the timbric similarity between songs. Observations and observation probabilities was modeled by rhythm descriptors and rhythm similarity. To query the model we simulate the insertion of the query song in the graph by calculating timbric and rhythmic similarity with the collection. A modified Viterbi algorithm was developed to extract semantically meaningful paths in this songs graph, from the query song. We infer the tags for the query by a weighted sum of tags from the songs in the extracted paths. We tested our model using the CAL500 dataset, a well known dataset for music information retrieval evaluation, and results are promising. Performance measures are good for an first implementation of a new model. This thesis is intended to show the results of our work between august, 2012 and march, 2013 on this project.